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A Tale of Two Numbers 

by Arthur & Margaret Snyder 



Comments will be forwarded. E mail: Clint Bradford 



Arthur Snyder is a physicist at Stanford Linear Accelerator Center. Margaret is a librarian at the 
Hoover Institution at Stanford University. She has been studying the Kennedy assassination since the 
late 1960s. 

A friend wrote me today to remind me that although David Mantik doesn't have an "open" Internet 
presence, that he is approachable. I found this out earlier this year, when he and I exchanged email 
messages...even when I was a little confrontational. David Mantik replied with respect, and even 
admitted a couple errors in one of his articles. 

Apparently Mantik has already privately admitted the errors described in the following article. 

- Clint Bradford, 12/11/98 



While reading Dr. Mantik's article ("How the Film of the Century was Edited") in Assassination 
Science. I was flabbergasted by the claim that the probability of the eyewitnesses to the "limo stop" 
being wrong was " 10' 17 " - a very small number. 

Tink [Josiah Thompson] called it loony. Anybody with some experience with eyewitness testimony 
knows it's never that good. 

This is what Dr. Mantik says in Assassination Science : 

"As a not entirely hypothetical example, if a single witness has only a 2% chance of being wrong (as 
in the table above [taken from Elizabeth Loftus' book. Eyewitness Testimony / ). and if 10 witnesses 
report the same event what is the probability that they are all wrong? This subatomic number is 
( 0.02 ) 10 =1(T 17 =0.00000000000000001. " 

10" 17 ? Wow! While I think it unlikely the Zapruder film has been extensively altered, I don't think it's 
that unlikely! 

So what are these two numbers? What do they mean? 

"0.02" is drawn from Table 3.1 in Elizabeth Loftus' book. Eyewitness Testimony . Here is the table: 
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Saliency 


Accuracy 


Completeness 


0 


61 


64 


1-12 


78 


81 


13-25 


81 


82 


26-50 


83 


92 


50-100 


98 


98 



Loftus adapted this table from a larger table in a paper by Marshall, Marquis and Oskamp (MMO). 
The paper was entitled, "Effects of Kind of Question and Atmosphere of Interrogation on Accuracy 
and Completeness of Testimony" (Harvard Law Review M. P- 1620). 

MMO were not attempting to determine the absolute accuracy of witnesses, but to gauge the relative 
effect of questioning modality and a "supportive" or "challenging" interrogation style. 



The MMO Experiment 

The MMO experiment consisted of showing a 2 minute movie to 151 subjects. The film is described as 
"fairly complex" but it is simple enough that I can outline it from memory. They counted about 884 
"items" present in their film. The events depicted are much simpler than what happened in Dealey 
Plaza 35 years ago. 

The MMO subjects were questioned immediately after being shown the film using one of four 
questioning modalities: 

1. Moderate guidance. 

2. High guidance. 

3. Multiple choice. 

4. Leading. 

How does this compare with the mixture of testimony, interviews and hearsay -- days, months and 
years later -- used for the witnesses cited in Assassination Science ? 

The table Dr. Mantik uses was constructed from the answers of those questioned using the multiple 
choice method! It is difficult to imagine a methodology further removed from the haphazard 
questioning of the JFK assassination witnesses than multiple choice. 

Dr. Mantik draws his 2% "chance of being wrong" (CBW) from the high saliency category in the 
Loftus table. He does not specify whether he is using the completeness or accuracy number and since 
they are numerically the same we can't tell. 
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The phrase "chance of being wrong" as used in Assassination Science is vague. Is he referring to 

a. Type I error or false negative (failing to see the limo stop when it really did stop). 

b. Type II error or false positive (seeing the limo stop when it didn't). 

In MMO's salience study it is the average number of times subjects either missed items in the film 
("completeness") or recalled them inaccurately ("accuracy"). The limousine stop is analogous to an 
"item" in the MMO film, so not seeing the stop would be a "completeness" error. This is what 
statisticians call a type I error — a false negative. 

MMO did ask a few questions that sought to elicit false positives, but salience refers to items actually 
in the film, so the most reasonable interpretation of the completeness numbers in this table is as false 
negatives. 

MMO give no information about correlations. E.g., did the subjects typically all get the same items 
wrong or were the errors evenly distributed among the items? Correlations can change probability 
calculations drastically. 

In the extreme case where all the subjects miss the same 2% of the items, the probability of them all 
missing a randomly selected item is 2% not (0.02) N . 

The "stop" witnesses are also correlated in another way. They talked to each other. In an extreme case 
the testimony of Marrion Baker is used both for himself (mistakenly) and for James Chaney (hearsay). 
The correlation between them is 100%. 



io 17 ? 

I find it difficult to express just how small this number is. For practical purposes it might as well be 
zero. Even somethin g as loony as film alteration is more probable than that! 

The important thing, however, is not the size of the number but what it means: 

10‘ 17 =(0.02) 10 is the probability that 10 witnesses each with an independent (uncorrelated) 2% 
probability of failin g to correctly observer an event (e.g., a "limo stop") would ALL fail to correctly 
observe it! It is the probability they would ALL say it had not stopped when it had stopped. 

It is NOT the probability of finding a group of 10 or so witnesses all saying the same wrong thing, 
e.g., the limo stopped when it did not stop. 

Hypothesis Testing 

There is a conflict between the eyewitnesses that say the limo "stopped" and the Zapruder film which 
does not show the limo stopping. 

We are considering two hypothesis to explain the conflict: 
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I. The film has been altered to remove the stop, 
n. The witnesses are mistaken. 

To compare these two hypothesis we need the probability that this conflict could have arisen under 
each hypothesis. 

For the film alteration hypothesis (I) the probability is small, but not computable. 

The relevant probability for the mistaken witnesses hypothesis (II) is the probability that a group of 
witnesses (large enough to catch our attention) would turn up. 

The probability that a specific group of witnesses is mistaken is irrelevant The relevant question is 
what is the probability of finding such a group. This probability is much larger. 

A Lottery 

It's like the lottery. The odds of winning are one in millions. The odds of any particular person w inning 
are very small, but the odds of somebody winning are high! 

The limo stop witnesses are the winners in a lottery to attract the attention of alteration advocates. 

The odds of any particular group winning are small; the odds that some group would win are much 
higher! 



False Positive 

Given that the limo did slow down (losing 1/3 its velocity in a fairly short time), and that the brake 
lights were on, it is not that surprising that the some people thought it stopped or nearly stopped. The 
MMO results have no bearingon the probability of this error being made. There is no reason to 
believe it is as small as 2 %. 

If there were 250 witnesses in a position to see the limo stop a false positive rate of 2 % gives an 
expected number of false stop reports of 5. The probability of 10 or more false reports would be about 
3%. Given the ambiguity of die situation the false positive rate could easily have been larger than 2%. 



Ex Post Facto 

Any probability analysis that is done after the fact is potentially subject to large selection biases. In an 
infamous case in particle physics a new particle was "discovered" (the £ (8.3)); the chance of it being 
background was calculated to be 10' 5 . It turned out to be selection bias plus a healthy statistical 
fluctuation. 

The often-cited London Times mystery death calculations make the same sort of error. 

The following biases should be considered: 
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• What would be accepted as a sufficiently striking disagreement with the Z-film to establish 
tampering? Many things other than "a stop" might have been considered to be in conflict with 
the film if they had turned up. 

• What witnesses count as saying the limo "stopped”? 

• What base of witnesses would have been expected to see the limo stop if it did? 

Self Destruction 

If the probability of JFK assassination witnesses to fail to see a stop that actually occurred were as 
small as 2% it would virtually rule out a stop having occurred! 

If there are 250 relevant witnesses then the expected number of witnesses that would fail to see the 
stop would be 5. The probability of 10 or more not seeing it would be about 3%. Many more than 10 
witnesses failed to see a stop ! Almost everybody would have seen the stop. 

The data is inconsistent with a 2% "chance of be wrong" and the limo actually stopping! 

Two Numbers, Five Mistakes 

• They calculate the wrong number. 10' 17 is the probability that a group of 10 would all be 
wrong, not the probability of finding 10 (or more) mistaken witnesses out of a larger group. A 
large combinatorial factor has been ignored. 

• Two different types of errors are confused: 

o Type I (the false negative), e.g., the probability of a group witnesses failing to see the 
limo stop when it did stop. 

o Type II (the false positive), e.g., the probability of finding witnesses that "see" the limo 
stop when it did NOT stop. 

• They ignore correlations. E.g., the brake lights were on, the police officers talked to each other. 

• The selection of categories was done ex post facto leading to large selection biases. E.g., "a 
stop" turned up among the eyewitnesses, but other equal striking discrepancies would have been 
latched on to if they had happened to occur. 

• The two percent "chance of being wrong" ripped out of context from Loftus' Eyewitness 
Testimony. It is not applicable to any real world situations much less the JFK assassination. 



• Loftus from Tink's talk: 

It is fair to say that salient details are remembered better than peripheral ones. Also, it is easier to 
mislead people about peripheral details. It is WRONG [Loftus emphasis], however, to say anything 
like 98% of salient details are accurately remembered. If that was shown in the Marshall case, it is 
only with those subjects, with that stimulus material, in that study. We virtually never make claims 
about absolute percentages because the real percentages in any situation depend on so many other 
factors. 
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. Tiiik was right! 
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